Conventional graphical user interfaces (“GUIs”) of computer software applications are typically a result of interaction between the software application and the underlying operating system (“OS”) of the machine in which the application is run. The OS provides mechanisms for applications to display information to the user through a hardware display. For example, in Microsoft® Windows based computer systems, applications interact with the Windows™ application programming interface (“API”) to display application output to a screen using OS window display elements, e.g., windows, buttons, menus, and the like.
Using this conventional approach, the microprocessor(s) in the computer system executes application code to produce application related results and additionally executes OS operations to generate display elements in which to place the application output. The microprocessor executes software instructions to generate the output graphics and sends them to the graphics hardware for display on the computer monitor or screen. Accordingly, conventional graphical user interfaces require significant computing resources to generate and display the output graphics associated with applications. In addition, the microprocessor is also tasked with executing the application functional code to produce the results that are then displayed.
These conventional GUI processes greatly reduce the processing speed for both the display of output graphics and the execution of the application functional code. This slowdown is most noticeable when multiple applications are executing simultaneously in a multi-tasking computer system. The OS-generated output windows for each of the active applications are being generated and refreshed through processes executing in the same microprocessor(s) that are executing all the functional code of the active applications. As the limits in the computing resources of the system are reached, the graphical display in the computer monitor shows the performance degradation by failing to properly refresh window content, creating ghost windows, failing to respond to user commands, and the like.
This type of performance degradation is most noticeable when multiple applications are executed that generate multi-media, graphics-rich output data. For example, digital photo or video software applications, e.g., video feeds, movies, or the like, consume much of the microprocessor computing resources and typically cause window operations, such as minimizing, resizing, and the like, to lag from the time the user input is received by the OS. Accordingly, it is desirable to provide a mechanism for the display of multiple output graphics offloading the output generation from the main processor.
In addition, conventional GUIs do not provide functional interactions with the display windows based on content relationships between the windows of different applications. Each application, through the OS API, generally controls the display of its user interface independently from other applications. The user may interact with the OS to physically move or resize windows within the display area of the monitor(s) but there is no mechanism provided in the OS or otherwise to exploit relationships between content being displayed in different windows from independent applications. It would be desirable to enable users to intuitively interact with displayed content by exploiting content relationships regardless of the source application responsible for generating the content.
The problems resulting from the current inability of conventional user interfaces to support management of the user interface environment in response to content relationships between different applications become particularly acute in enterprises that utilize many different sources of data and applications to make business, analytical, financial, emergency, health, or other types of complex decisions. For example, it is not unusual for a business to have multiple different databases managing its inventory, accounting, sales, manufacturing, and technical support systems, as well as completely unrelated applications for accessing such data. A user in such an enterprise will typically have open on his or her computer numerous different ones of these applications while attempting to make business decisions. Conventional operating systems such as Windows™ leave it entirely to the user's own capabilities and decisions to manage where to place the windows for these applications on the screen, which ones to attend to, and how reorganize or reconfigure the application windows in response to changing information. For example, if the user is responsible to update a sales report in response to increases in sales data, the user must monitors sales database in one window, and when the sales data reaches a certain threshold, manually select a reporting application window to update a report.
Accordingly, what is needed is a user-centric graphical user interface that leverages processing capabilities of the graphics hardware and off-system image pre-preprocessing to offload some of the graphics display processing from the microprocessor in the user's device and provides application and hardware independent functionality for content-driven organization and presentation of content to the user.